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PREFACE 


I f a camera takes a picture and no one notices, did it really hap¬ 
pen? In this book, we’re going to show you how to make a work¬ 
ing Raspberry Pi-based camera system so that you can capture 
time-lapse images and view via WiFi, trigger the camera if motion 
is detected, and even carry out basic facial recognition as an 
introduction to machine learning methods. 

Take a Raspberry Pi and add a camera module, and you have 
a programmable camera. Add some software, and you can start 
to do interesting surveillance and automatic object recognition 
work with it. Activate the Pi as a WiFi node and you can do all 
these wonderful things from a distance. 

A good surveillance system does more than take pictures. It 
should also turn those pictures into actionable information that 
increases your knowledge. That’s now easily done in software, 
and we’re going to show you how. 

This book pulls together a set of little tricks—setting up Pi 
cameras, making a Pi broadcast as a WiFi device, adding time 
lapse and motion detection and face recognition, and sticking a 
battery pack on it so it can function anywhere—to create porta¬ 
ble spy cameras. We’ve used these rigs in everything from “Find 
the Pi” party contests, to implementing privacy-respecting secu¬ 
rity in our lab, to showing off modern tech like facial recognition. 
Other uses could include monitoring deer and wildlife, checking 
your house mailbox for mail arrival, and capturing time-lapse 
sequences of natural events or traffic patterns. 

Our first project will be to create a portable “SpyPi” camera 
rig that broadcasts images via WiFi without needing a network— 
you just have to log into it to see what the camera sees. It’s great 
for a “find the camera” hunting challenge or stand-alone security. 



The second project will involve setting up a time-lapse cam¬ 
era that can do simple motion detection—perfect for monitor¬ 
ing a location. The system will include automatic deletion of old 
images after a week. 

The third project will be to add facial detection and facial 
recognition to your SpyPi. In the process, you’ll learn about the 
tools and software installed by the end of these projects so that 
you can go beyond this book and explore additional machine 
learning methods. 

So grab a Pi, an SD card, a USB camera, and a USB battery 
pack, and let’s go! 
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Introduction to the 
Raspberry Pi 


In this book, we will be discussing how to take a Raspberry Pi 
and turn it into a do-it-yourself security system. The following 
chapters will cover projects ranging from taking time-lapse pho¬ 
tos to using facial recognition. Before we get to any of those, 
though, we need to spend some time preparing the Pi so that it’s 
ready for those projects. 

These setup processes may not be the most exciting informa¬ 
tion covered in this book, but they are arguably the most crucial 
for you to know. The Raspberry Pi is a versatile device, providing 
a good base for a variety of projects. It is for that very reason 
that knowing how to modify and customize your Pi to suit a given 
project is critical. 

This chapter will cover, among other things, how to install 
software packages and connect to your Pi remotely. This knowl¬ 
edge is useful in any Pi project and helps open up possibilities 
for taking your projects further. 



FIGURE 1.1: Raspberry Pi 3 with USB webcam 


GEAR 

You will need the following: 

* A Raspberry Pi (the Pi 3 model is recommended, but any 
model will work)—$35 

* A microSD card (8GB minimum)—$6-$20 

* A USB camera (or Pi-specific ribbon camera)—$10-$20 

* A 5V battery pack (cell phone recharger, USB charge pack, 
etc.)—$5-$10 

* A USB WiFi module (only if you’re not using Pi 3 or Pi Zero 
models, which have built-in WiFi)—$10—$15 

For the initial setup, you will also need 

* HDMI monitor and cable 

* 5V Pi-compatible power supply (a microUSB phone charger 
should work) 

* USB mouse and keyboard 

The following are strongly recommended but not mandatory: 

* A USB power supply (2000mA or more)—$10 

* A case—$5, or 3D-print your own with plans from 
thingiverse.com 
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raspberrypi.org lists online sites by country, so that’s a good 
place to start. You can also buy kits that include everything listed 
(and often more add-ons). 

To make your project portable, you'll need a standard USB 
battery pack (the kind used to recharge phones and devices; 
you can find them for $5-$10 at electronics or convenience 
stores). The larger the pack, the longer the Pi will last. A five- 
dollar 4000mAH (milliamp hours) device powered our project 
for over four hours at an outdoor conference. 

For the SD card, we recommend you get a name brand like 
SanDisk, Kingston, PNY, Sony, Toshiba, or Samsung. Don’t get 


Older Pi Models 

Just to complicate things, if you have an older Pi Model A or B 
(not an A+ or B+ or Pi 2 or Pi 3 or Pi Zero), it will require the 
older, larger SD instead of a microSD card, but at this point the 
older Pis are hard to find so you probably don’t have to worry 
about it. The rule is simple: look at what physical form factor 
card (SD or microSD) fits in your Pi, and buy that one. Typically, 
you want to buy the latest model, which as of this writing is the 
Pi 3 Model B+, because the newest model will be faster and have 
more memory yet will be the same price as the other models. 
But you can use an older one, especially if you already have it 
lying around. There are different models of Pi. These are all fine: 
Pi 3 Model B or Model B+, Pi 2 Model B, or even Pi 1 Model B+- 
they all have four USB plugs so they’re easy to hook up to lots 
of things. Avoid the “weird” ones (Pi Zero, Pi Compute Module, 
or the old Model A) unless you really know about Pi. But it’s easy 
to tell them apart—buy a $35 Pi and you’re likely good. Fun fact: 
Raspberry Pis are always $35 because there’s only one vendor 
and Raspberry Pis are intended for educational and maker use, 
so they don’t change in price. Since all the models we can use 
are the same price, there’s no reason not to buy the latest. 
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a cheap generic card; they aren’t as reliable. The card must be 
marked as SDHC (secure digital high capacity). The cards are 
available at speeds of Class 2, 4, 6, or 10—the higher the num¬ 
ber, the faster the card. As long as it’s at least Class 4, the actual 
speed isn’t very important—the Raspberry Pi Foundation recom¬ 
mends Class 6 or higher, but Class 4 work well for everything in 
this book. For more details, you can visit www.raspberrypi.org/ 
documentation/installation/sd-cards.md. 

For the camera, if it’s USB and it works on your PC or Mac or 
laptop, it’ll most likely work on a Pi (we tested 11 different cam¬ 
eras, including super-cheap generic ones!). Cameras for surveil¬ 
lance don’t need to be fancy; they are just commodities. It’s the 
software you’ll be installing that does the magic. 

To begin, you’re going to hook up your Pi to a monitor, key¬ 
board, and mouse, and use it like an ordinary computer. After 
you finish getting everything set up and are ready to deploy the 
device, you’ll ditch the keyboard, mouse, and HDMI monitor and 
power cable; add the battery pack; and shift to “headless” mode, 
where the Pi is accessible only via the Internet. In this mode, 
you'll interact with the Pi via a web browser, remote access tools 
like VLC, or an SSH terminal. 

We’ll cover a base loadout and setup, and we’ll include side- 
bars on using other forms of gear. For example, any Pi models 
(including the Pi Zero or original Pi Model A) can do everything 
in this book; they just require a little extra setup. As for cam¬ 
eras, a Pi can use either USB cameras, its own model of ribbon- 
cable cameras, or special cameras that connect via the general- 
purpose input/output (GPIO) pins that stick off the Pi board. As 
we said, we recommend an inexpensive USB camera that you can 
buy just about anywhere, but we’ll include notes on using other 
cameras if you want to play around with alternatives. 
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SETTING UP 


Two ways to set up your Pi (based on the official Raspberry Pi 
Software Guide at www.raspberrypi.org/learning/software-guide) 
are as follows: 

* Buy a preloaded SD card with NOOBS (New Out-of-Box 
Software) or Raspbian. 

* Buy a blank SD card and load either NOOBS or Raspbian 
from http://raspberrypi.org/downloads. 

Both lead to the same result—you first boot your new Pi with 
your microSD card that has the operating system installed, and 
then you follow its installation menu. Pro tip: If you’re in the 
United States, you’ll have to reset the keyboard from UK style 
to US style or else you’ll find all the punctuation marks are in 
the wrong place (on the Pi, type sudo raspi-config and use the 
location or keyboard settings menu to do this). 

You’ll need to hook up the Pi to an HDMI monitor as well as a 
keyboard and mouse for setup. You can also plug in a wired net¬ 
work cable, or just wait and configure the built-in WiFi (Pi Model 
3 or later, or add in a WiFi USB dongle if you have an earlier Pi 
without built-in WiFi) to connect to your local network. Once 
we finish this, you won’t need to use the monitor, keyboard, and 
mouse to work with it, which is a great advantage of a Raspberry 
Pi. You can work headless, with just the Pi itself and a power 
source, wirelessly and with no extra attachments needed. 


Plug in the USB camera, ideally before you power up 
the system. The Pi will auto-detect the camera upon boot; it 
might not always detect the camera if you plug it in or remove 
it while the Pi is running. 
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ADDING SOFTWARE 


We’ll install all the free software you’ll need for this book at once. 
Also, we’re not going to use the GUI or have you click any icons; 
instead, you'll run commands in the terminal by choosing Termi¬ 
nal from the Start menu. (If you didn’t install a GUI, you’re already 
in the terminal and ready to go.) 

The command for installing software in Raspbian Linux is 
sudo apt-get install softwarename , which automatically finds and 
installs the software via the Internet. 

First make sure your operating system is current and up to 
date by entering the following in the terminal on your Pi: 

sudo apt-get upgrade 



FIGURE 1.2: sudo apt-get update 


Next, install the software you’ll need to complete Chapter 1: 

sudo apt-get install fswebcam -y 
sudo apt-get install feh -y 

sudo apt-get install python-pi camera python3-picamera 
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pi@raspberrypi:/ S sudo apt-get install fswebcam 
Reading package lists... Done 
Building dependency tree 
Reading state information... Done 

fswebcam is already the newest version (20140113-1+bl). 
FIGURE 1.3: Installing fswebcam 


Then, install the software for Chapter 2: 

Continue with the Chapter 3 software next: 

sudo apt-get install motion -y 

Finally, install the Chapter 4 software: 

sudo apt-get install python-opencv -y 

sudo apt-get install python-pip python3-pip -y 

pip install numpy 

Keep that terminal window open, because you'll type more 
commands in it. As a maker/hacker, you’ll find it’s quicker to type 
and edit in this window rather than use a GUI. 


SETTING UP THE CAMERA 

Time to plug in your USB camera and do some initial diagnostics. 
To capture images, you'll use fswebcam as the primary program. 
For more details (beyond our usage in this book), you can learn 
more at www.raspberrypi.org/documentation/usage/webcams, 
but let’s skip right to the real work. 

When you typed sudo apt-get install fswebcam in the termi¬ 
nal, you installed a package of programs that can control a USB 
camera. You need to test them to make sure that they installed 
correctly. Type the following in the terminal to check: 

fswebcam image.jpg 
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pi@raspberrypi:- S fswebcam image.jpg 
- - - Opening /dev/videoO... 

Trying source module v412... 

/dev/videoO opened. 

No input was specified, using the first. 
Adjusting resolution from 384x288 to 424x240. 

--- Capturing frame... 

Captured frame in 0.00 seconds. 

--- Processing captured image... 

Writing JPEG image to 'image.jpg'. 

FIGURE 1.4: Taking a test image with fswebcam 


This command will tell the camera to take a snapshot of what¬ 
ever it sees and save it to the current directory. If you are using 
the GUI, you should see the image once you open the File Man¬ 
ager. If you are working via the terminal, simply check the direc¬ 
tory with the Is (list) command. 

pi@raspbarrypi:- S Is 

Desktop Downloads Music Public Templates 

Documents image.jpg Pictures python_games Videos 

FIGURE 1.5: Finding the test image in the current directory 


“Timed Out” Error 

If fswebcam isn’t working and you’re getting the error message 
Timed out waiting for frame! No frames captured, we have 
a solution. The problem is that the system can’t tell what reso¬ 
lution the camera is. To fix this, just force the resolution of the 
camera in any call of fswebcam as follows: 
fswebcam -r 320x240 image.jpg 


If you are working through the Pi’s GUI, it’s a good idea to 
open the test image and check to make sure that it is passable. 
Not all cameras are the same in terms of quality, and a cheaper 
model may be more prone to taking unusable photos. That being 
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said, once you see your image and are able to recognize what the 
image is of, your camera should be good enough for the next step. 



FIGURE 1.6: Test image captured with fswebcam 


Overexposed 




A common problem found when using fswebcam with cheaper 
USB cameras is that the photos taken have a tendency to be 
overexposed. Fortunately, fswebcam is capable of automati¬ 
cally adjusting its exposure by taking multiple frames. To tell 
fswebcam to do this, use the following: 


where n is the number of frames fswebcam is to take. Most people 
recommend taking 50 frames, though as few as 20 frames 
should result in passable images without taking up too much 
processing time. 


GOING HEADLESS: ACCESSING 
YOUR PI 

All Raspberry Pis start with the default network name/ID of 
raspberrypi, but you’ll change your machine name. This makes it 
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easier to find (in case other people stick a Raspberry Pi on the 
same network) and more secure (since people won’t automati¬ 
cally know the name of your machine). Start up the configuration 
program: 

Click Change Hostname and give your machine a new name. 
We suggest spypi, and that’s the example name we’ll use in this 
book. Once you set this, it always stays that way. And now, on the 
network, your machine can be found as spypi .local (thanks to a 
built-in package called Avahi, if you were curious). 

So instead of going to, say, Google or Makermedia.com, you 
can point web browsers and other tools to spypi .local and find 
it. Bear in mind this is only local-, you won’t be able to connect to 
your Raspberry Pi if you’re not on the same home or work net¬ 
work you plugged your Pi into. The rest of the Internet can’t see 
your PI, and that’s just what you want—private, not public, access. 

We’ll also make your Pi remotely accessible, so even if it isn’t 
hooked up to a monitor and keyboard you can still access it from 
any Internet-connected device on the network with an SSH client 
(on Windows, PuTTY; on Mac/Linux, ssh) as long as you set the 
Pi to accept that. On the Pi, run sudo raspi-config and choose 
Interfaces, then choose SSH and “Yes, enable SSH.” This allows 
remote logins from authenticated users who know the Pi user- 
name and password (as you do). The PuTTY or ssh client will let 
you remotely access your Pi, removing the need for a keyboard, 
mouse, and monitor. 

To log into your Pi remotely, you’ll use SSH: 

* In Windows: Install PuTTY from putty.org (freeware), and 
then run the program. Enter spypi .local as the machine you 
want to log into, with the username pi. 
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* In macOS or Linux: Call up a terminal and log in with the 
following: 

ssh -X pigspypi.local 


PuTTY Configuration 



FIGURE 1.7: Connecting with PuTTY 


In either case, you’ll be prompted for your pi password. You 
can change that password on your Pi by again entering sudo 
raspi-config and using the Change Password option. (Part of 
the ease of the Pi is that controlling the various options is pretty 
simple.) 


Using raspi-config 


Perhaps in the future, beyond this book, you might get into add¬ 
ing attachments and wires to your Pi. When you add switches, 
sensors, motors, and other systems, the instructions will gen¬ 
erally say something like “Enable the PC interface” or “Make 
the serial port accessible.” Again, those are just options in the 
raspi-config tool that you can turn on or off. They are turned 
off by default to make your Pi a little more secure. A core rule 
of computer security is to not turn on a service or software or 
port if you aren’t actively using it, because anything “open” is 
also a potential opening a malicious program can exploit. The Pi 
smartly keeps most options turned off, and then lets you easily 
toggle options with raspi-config as needed. 
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Multiple Cameras 




Yes, you can put more than one USB camera on a Pi—as many 
as there are open USB ports, in fact. The fswebcam program 
defaults to the camera it finds using the device location /dev/ 
video©, but you can add other cameras. Once they are plugged 
in and (hopefully) auto-recognized, see which ones exist by 
entering 
Is /dev/video* 


Sample output for two cameras might look like 

/dev/videoS /dev/videol 


And now you can use fswebcam to point to a specific camera 
like this: 


Using the Pi Ribbon Camera 


You can buy a special camera just for the Pi and use it instead 
of a USB webcam. It plugs into the Pi using a ribbon cable. To 
access it, instead of using fswebcam you can use the built-in util¬ 
ities raspistill (for single pictures) and raspivid (for video). 


V 


J 
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Time-Lapse 
Imaging Viewed 
via the Web 


T he official title of this project is “A Surveillance Pi with Time- 
Lapse Image Capture Accessed Remotely via the Web,” but we’ll 
stick with the SpyPi name. In the first chapter we got the Rasp¬ 
berry Pi to produce a single image; now we’re going to have it do 
some bona fide surveillance. You’ll set up your SpyPi to automat¬ 
ically take a picture every minute and also make a web page so 
that you can remotely access the most current image at any time. 
And we’ll talk about how you can combine the images to make a 
single time-lapse animated GIF of the activity! 

You'll create all of this via the terminal, as in the previous 
chapter. In this case, you’ll write scripts using the nano editor, 
save them, and then run them with the Python language inter¬ 
preter. Meanwhile, you’ll set up the web server in the background. 
Finally, you’ll use a stand-alone tool to make the animated GIF. 


A typical use case would be to set up your Pi in a forest or 
over an area where you need to maintain security, or even to 
capture traffic at a busy street intersection. Really, the legal uses 
are up to you. Everything we make is configurable, so you can 
make your system take one picture of road traffic every second, 
or one picture of the sunset every minute, or one picture of your 
house every hour. Best of all, you don’t need to be near your Pi to 
view these images; just point your web browser to the Raspberry 
Pi’s web page and watch the images from the luxury of your own 
home. 



FIGURE 2.1: Time series of a SpyPi surveillance of our basement floor, 
showing a stuffed animal crossing. (Image brightness is due to camera.) 


CAPTURING TIME SEQUENCES 
AUTOMATICALLY 

First, you'll write a short script file using the Python language to 
automatically run your camera with a regular time sequence. To 
do this, you’ll use the nano editor to enter the program. Type the 
following commands into the terminal: 

And enter the following code exactly as you see it. Python 
is very touchy about indentations—be sure to indent the code 
exactly as shown here. We’ll use 60 seconds (1 minute) as a delay 
time, but you can change that value in the code to whatever you 
need. Lines preceded by # are comments that explain the code. 

# Spy-Pi code version 1 

# this code takes an image every minute (60 seconds), 

# and overwrites the current image with the new one 
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while Tr 



I Exit B Read File E Realace fl Uncut TextS To Linter B Go To Line 

FIGURE 2.2: What you’ll see in nano when you make this first Python script 

This is a general structure we’ll use often in this book—using 
Python as a wrapper around our external programs like fswebcam. 
We’ll later add in a package called openCV for facial recognition. 
Python is easy to learn and modify, and it will let us set time-lapse 
loops and other tricks. 

Once you've typed in this program, save it by pressing Ctrl-X 
and name it t-imelapse.py. At the terminal prompt run it with 
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python timelapse.py. Your gadget is now taking a new picture 
every 10 seconds, overwriting the old one. You can see the image 
on the desktop or look at it within the window by typing feh 
spycam.jpg. 



FIGURE 2.3: Our first image: a stuffed animal on a piano! 

This code will run forever, as long as the Raspberry Pi has 
power. You can stop it at any time, fortunately, by pressing Ctrl-C. 
There’s also a fancy way to run it in the background so that you 
can type other things in the terminal—when you call it up, add 
an &, as in 

python timelapse.py & 

Don’t do this yet. In the next chapter, we’ll talk about using 
a routine called crontab to automatically run your code in the 
background. If you do want to play around with using & to run the 
program in the background, know that if you type fg it will bring 
the program up again, at which point you can then press Ctrl-C 
and kill it. Playing around with running scripts in the background 
or bringing them to the foreground so you can nix them is a very 
common Pi/Linux way to work on and run scripts. 

MAKE THE PI INTO A WEB SERVER 

To make your Pi into a web server so that you can point a browser 
to it, you...well, actually, you already did this. Back in Chapter 1 
when you typed sudo apt-get install apache2 -y you told the 
Raspberry Pi to install a complete web server package. We’re just 
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going to make it easier to modify things. (You can also read more at 
www.raspberrypi.org/documentation/remote-access/web-server/ 
apacbe.md.) 

Here’s your default homepage, if you didn’t change your Pi's 
name: 

http-.//raspberrypi.local/ 

If you did change it as we recommended, it’s 
http://spypi.bcal/ 

and, if you are on your Pi, it can also see the page as 
http://localhost/ 



Time- 


Imaging Viewed via the Web 































To make life easier, we’ll show you how to make a link to where 
the Pi stores web pages so it will be easy for you to drop stuff into 
your home directory. Type these two lines 

In -s /var/www/html /home/pi/web 
sudo chown pi . * 

to make a new directory in your home directory called web, which 
is the location where any web page is stored. The chown line 
makes it so you can write and edit stuff in that (otherwise, it’ll tell 
you permission denied). Anything you put into the web directory 
will be visible to the outside world via a browser, if the individual 
knows the address of your Raspberry Pi. There’s already a default 
page called index.html there. Let’s change it! 

Go to the web directory: 


Move the original index.html file to index-orig.html: 

mv index.html index-orig.html 

And create a new index.html file: 

nano index.html 

And now that you've invoked the nano editor, type the fol¬ 
lowing into the file: 

and then save the file and look at it in a browser. 

<r ' 0 <Zt \<K spypi.local 

Hi 


FIGURE 2.5: The world’s simplest web page 
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Okay, it’s a very boring web page, but as you can see, any¬ 
thing you put into index.html appears to your web visitors. 

Now you’ll make a new index page that works with the cam¬ 
era. You’ll also make a new Python script that automatically trig¬ 
gers the camera. Together, the two work like this: Every 1 minute, 
the Python script tells the Pi “Take a new picture and stick it in 
the web directory.” Meanwhile, people visiting your new Pi web 
page will see the most recent picture—and their browser will 
automatically refresh itself every minute so it can catch when 
the picture changes. 

Also, that 1 minute is an arbitrary time setting that you can 
change. If you think what the Pi can see will change faster or 
slower, make it a different time (every 5 minutes, every 30 sec¬ 
onds, every 5 seconds). Similarly, you can change the web 
page to update more rapidly—but beware, it takes time to load 
an image over the network via the web, so any web page that 
refreshes faster than 1 minute might mean your browser hasn’t 
even finished loading a picture before the next refresh happens. 
Put another way, you should not take pictures faster than you 
can serve them. 

First, modify our previous timelapse.py script by changing the 
one line that reads 

directory = '/home/pi/' 

to instead read 

directory = 1 /home/pi/web/' 

This will make the camera program write the image file into the 
web-visible area. 

We now need to put an HTML web file in that web area that 
points to the picture. At the terminal prompt, type these two lines: 

nano index.html 
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and then, when nano comes up, type this in the editor: 


<headxtitle>SpyPi </title> 
<meta http-equiv="refresh" 

<img src='spycam.jpg' alt=' 


</body> 

</html> 


content="60" /> 




Now save this code (in nano, press Ctrl-X and name the file 
jndex.html). Feel free to change SpyPi in the title and the image 
refreshes every minute! text. That’s just text for this demo and 
it can be changed to read anything you want. The key part is 
the meta keyword in the head tag, which sets the auto-refresh 
to every 60 seconds. This triggers any visiting web browser to 
reload the page every minute. You can set any time interval; one 
minute is useful for tracking slow state changes such as the cam¬ 
era being moved. Set it longer if you find it takes a long time to 
load an image in your browser. 

Let's give it a try. Start the infinitely looping Python script: 

python timelapse.py 

and look at what’s in the web directory by pointing your smart¬ 
phone or tablet to the web page http://spy pi'./oca//. 

You should see what the Pi sees in your web browser. It’s the 
same test image as before, but now you can access it via your 
web browser from a remote machine. Success! 
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FIGURE 2.6: Sample image from our SpyPi, framed in your web browser 


THE PI AS A REMOTE WIFI HOTSPOT 

Right now, the Pi is running the web server and technically, the web 
pages are visible to any other machine on its network. You can find 
it from other machines via a web browser at http://spypi.local/. 

This is great for a SpyPi that is able to log into your local WiFi 
network. You can use the SpyPi at home or work, and you’ll be 
able to check it from any machine on that network. You can place 
the Pi so that it creates its own hotspot that enables people to 
log into it, even if the Pi can’t connect to an external network. 
This is great if you want to, say, place it in a forest to monitor 
wildlife and still want to be able to log into it remotely. 
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The “Find the Pi” Game 


With the basic rig you’ve built, you now can play Find the Pi. 
One person hides the Raspberry Pi rig—the Pi, camera, and 
power supply, powered up and serving images—somewhere in 
a given area, such as a park, a school, or a backyard, for instance. 
The other players log into it with their smartphones or tablets 
and, based purely on the image they see from the Pi, try to 
figure out what the Pi is looking at and therefore where the Pi 
must be hidden. 

Good hiding places should be somewhat generic: players must 
be able to see some clue that creates a puzzle. For example, 
a totally dark image is useless, but a Pi that can barely see an 
“exit” sign or indoor plant gives players a chance. 

The person who finds the Pi can then choose a new hiding place 
for it, and the game continues. Here’s an example of a picture 
that makes you think, “I bet I could walk around the author’s 
basement and figure out where the Pi is looking out.” 


j - ==r= -osas 
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In this section, we’re going to set up the Raspberry Pi so it 
doesn’t have to be connected to an existing network. Instead, 
the Pi itself will create a WiFi hotspot that you can log into. This 
means anyone close enough to be in WiFi range of the Pi can 
connect to it. 

To access your Pi, even without an Internet connection, follow 
any of these three tutorials: 

www.raspberrypi.org/documentation/configuration/wireless/ 

access-point.md 

www.instructables.com/id/Make-Your-Pi-a-Local-Cloud-Server/ 

https://thepi.io/how-to-use-your-raspberry-pi-as-a-wireless- 

access-point/ 

The software installed is two packages called HostAPD (that 
makes your Pi a stand-alone Internet device) and dhcp (a net¬ 
work standard for automatically assigning addresses and names 
to devices that connect to your Pi). We’ll also toss in a firewall 
called iptables. You'll install them with the usual: 

sudo apt-get install hostapd isc-dhcp-server iptables-persistent 

and then follow any of the tutorial links. Once done, you can log 
into your Pi as if it were its own Internet. 


CLEANING AND ARCHIVING 

At this point you are always serving up the current image. How¬ 
ever, archiving past images moves the SpyPi past “toy” into “gen¬ 
uine surveillance." Let’s modify our Python script to save the 
previous N days’ worth of images, where n is however many days 
you want to keep. 
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Type nano timelapse2.py in the terminal, and enter the follow¬ 
ing code as is. Remember to maintain the proper indentations. 


# Spy-Pi co 

# it also it 


ute is 13,080 files per w 
3 / delay 

o put the files, and the 


* also, this is the file the 



ig Raspberry Pi Vision 




Again, running python timelapse2.py will do three things: 

1. Take a picture at the given interval (here, 1 minute). 

2 . Save all the pictures it takes, up to the given period (here, 1 
week), deleting older ones. 

3 . As written, it restarts the image count at 0 whenever you 
power up, so you lose old data and start afresh each time. 

You may want to clean up when you stop or restart the time- 
lapse routine. These two commands, typed into the terminal, will 
get rid of all old files: 

cd web 

rm spycam*.j pg 

Now you can rerun python timelapse.py without the clutter 
from previous runs. 


Using the Pi Ribbon Camera? 

If you are using the Pi ribbon camera instead of a USB web cam¬ 
era, just change the call from fswebcam (which automatically 
finds the USB camera) to raspi still (which automatically uses 
the Pi ribbon camera). 


ANIMATING ATIME-LAPSE GIF 

To view the archive of images, log into your Pi and either down¬ 
load the set or view the images on the Pi. Or you can choose to 
make them into an animated time-lapse GIF! Type in your termi¬ 
nal the following sequence to build a GIF at any time. 
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First, if you didn’t install imagemagick in Chapter 1, type this: 

sudo apt-get install imagemagick 

Now, using your terminal, type these commands to go into 
the directory with the files and turn all the images into a time- 
lapse animated GIF file. Note the unit: delay is in hundredths of 
a second, so 10 = 0.1 second between frames, and loop 0 = loop 
infinitely: 

cd web 

convert -delay 10 -loop 0 spycam*.jpg timelapse.gif 

Let’s look at it via the web. As before, you need to make 
a web page that calls it. So in the web directory, make the file 
timelapse. html: 

nano timelapse.html 

In that file, put these lines: 

<html><body> 

</bodyx/html> 

Now, with your web browser, point to 
http://spypi.hcal/timelapse.html 

And you can see your amazing animated time-lapse image! 
The following is a tiled version of eight time-lapse images. 
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Pi Motion 
Detection 


I n this chapter, we will be discussing how to turn your Raspberry 
Pi into a motion-detecting surveillance system. This is simpler to 
accomplish than it sounds, since the software we need is ready 
in a downloadable package for the Pi. Once it’s installed, we can 
dive right into operating and fine-tuning the program. 

As with the previous project, we’ll be working mostly from 
within the terminal, though this chapter will cover how to view 
the program’s output with the GUI. We will be using the nano 
editor to help customize our program to suit our needs. 

Motion detection allows for more selective and sophisticated 
image capturing than just taking a photo every 10 seconds. Now 
you can set up a wildlife camera that takes routine snapshots and 
automatically records video whenever an animal walks by. We’ll 
also show you how to set up filters so you can quickly find who’s 
been knocking on your front door without having to look through 
scores of images with the same view of your welcome mat. Last 
but not least, you can connect your motion-detection system to 
the Internet and watch what it sees wherever you are! 


WHAT IS MOTION DETECTION? 


Motion detection, in essence, is the search for changes in an 
environment caused by an object moving. A program does this 
by comparing two sets of input data about the environment and 
deciding whether there is a significant difference between them. 
In this chapter, we will compare literal images to try to detect 
whenever something has moved. 

It’s important to understand how one image is compared to 
the last taken. Essentially, the program counts how many pixels 
have changed since the last image. With this in mind, we must set 
a threshold for the differences to be reported. If the threshold is 
set at a low percentage of changed pixels, then the system will 
flag minor events like branches or papers rustling in a breeze as 
movement. If the threshold is set higher, it will take a more drastic 
change to count as movement. 




FIGURE 3.2: Environment after movement 
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But not all changes to an image are caused by movement. 
Suppose your camera is looking at a deserted street at twilight. 
At the appropriate moment, the street lights will flicker to life. 
The scene will change drastically—formerly dark shadows will 
become warm pools of light. But does this count as movement? 

This is where sensitivity comes in. Sensitivity refers to certain 
parameters designed to help reduce the number of false posi¬ 
tives reported by the system. If sensitivity is not taken into con¬ 
sideration, the system will report any time there’s any significant 
change between images. This is fine when someone walks in or 
out of a room—but not when the light in the room is turned off. 
We can screen out such events by including a contrast setting 
with the camera. 


HARDWARE 

This chapter uses the same hardware listed in Chapter 1. 


SET UP MOTION DETECTION 


You will need to download the software needed for motion 
detection. If you didn’t do so in Chapter 1, enter this command: 

While the packages are downloading, a prompt will come up 
asking if you wish to continue with the installation. Just press Y 
and then Enter to finish the download. 

TESTING MOTION WITH GUI 

The motion package is fairly ready to use once it's installed. The 
easiest way to test whether the program is operational is to work 
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using the Raspberry GUI. To get the package running, first enter 
the command: 


This command will start motion and have it record images. To 
view what your camera is seeing, open the Pi’s web browser and 
type in the address http://iocaihost:808i. This will establish a 
connection to motion and display its images in the browser. 



FIGURE 3.3: Streaming motion from localhost 

When you first connect to the service, don’t be alarmed if 
the images are small or the frame rate seems slow. These are the 
default settings assigned to motion. We'll go over how to recon¬ 
figure the service in the next section. 

TESTING MOTION WITH THE 
COMMAND LINE 

If you are only using the command line, testing motion isn’t too 
complicated. The first thing you need to do is start motion: 
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Now, because you don’t have the web browser to watch the 
live feed from the camera, let the program run for a few seconds 
before signaling it to stop. After running motion for about 30 
seconds, enter the following to kill the program: 


Once motion is stopped, you can then try to look up the 
images it took while it was running. By default, motion should 
save its images to the current working directory. To check, type 
the following: 

ts 

Once you execute this command, you should see a list of 
JPEG files. Most are named with a time code followed by the 
word snapshot. These are the snapshot images that motion takes 
at a set rate to look for changes. There should also be an image 
named lastsnap.jpg. This is the latest image taken by motion and 
is updated each time motion captures a new photo. It is also what 
you see when viewing motion in the browser. 

The point of this test is to make sure motion is running as it 
should in its default state. If the program is displaying what it sees 
in the browser and it is writing files as described, then that means 
motion is running fine. This test also acts as a quick introduction into 
how to operate the program. You’re ready to move to the next step! 

CUSTOMIZING MOTION 

We mentioned earlier that it is possible to customize the settings 
in motion in order to have the program suit your needs. Fortu¬ 
nately, almost all these changes can be made and applied in a 
single file: motion.conf. 

To access motion.conf, open the file using nano: 

sudo nano /etc/motion/motion.conf 
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The configuration file for motion is lengthy to say the least, 
but don’t be daunted. The different settings are documented and 
explained within the file, and if you need additional information, 
you can find plenty of online resources. 

One of the perks of using nano to look at the file is that you 
can search for keywords by pressing Ctrl-W. This makes navigat¬ 
ing the file and finding specific settings much easier than having 
to scroll down through the various configurations. 

Let’s focus on some of the more basic options available. 


NOTE It is important to keep track of changes you make in 
the configuration file. Needs vary from project to project and 
mistakes are always a possibility, so unless you like the idea of 
combing through the entire file for that one line of code, a record 
of what you changed and what you changed it from is a good 
thing to have on hand. 
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A good place to start in motion.conf is the height and width 
values. As you saw, the default image in the browser is a bit on 
the small side. That’s because the default values of height and 
width are 240 and 320 pixels, respectively. If you want to view a 
larger image, you simply need to replace those values with larger 
numbers. 


NOTE To avoid weird dimensions with your images, you 
should maintain the ratio between height and width (3:4). Also, 
the valid range of dimensions is limited by your camera: you’re 
never going to get a 1280x960 image from a camera with a 
maximum resolution of 640*480. 


For practice, let’s double height and width. Find those lines in 
the file and edit them to say 

width 640 
height 480 

Once you've specified the new dimensions, exit the file to test 
them. To save the changes you made, press Ctrl-X followed by 
Y, and then press Enter to reach the command line. Run motion 
as you did in the previous section and see whether the image is 
larger. 

If the image in the browser is the same as before, you may 
need to reboot the Pi to have the new settings take effect. To 
initiate a reboot, at the command line type 


Once the Pi is rebooted and you are logged back in, you 
should be able to start motion and see the changes. 

One other value we should look at is threshold. The threshold 
value determines how many pixels need to have changed 
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between two snapshots to trigger an event. By default, threshold 
is set to trigger if 1,500 pixels have been changed. 

Now recall what we said about the main principles of motion 
detection at the beginning of this chapter. Although 1,500 may 
sound like a high number, that is 1,500 pixels out of the entire 
image. Using the default settings in motion, let’s calculate what 
percentage 1,500 is of a 240 x 320 image: 

240 x 320 = 76,800 
1,500 / 76,800 = 0.02 

Using the default settings, motion triggers an event whenever 
there is a 2 percent difference between photos. That is far too 
sensitive! With a threshold that low, an event could be triggered 
by a fly buzzing by the camera. The sensitivity is increased even 
more for a 480 x 640 image: 1,500 pixels represents a threshold 
of 0.5 percent. 

To make motion less sensitive and to help avoid drowning in 
false alarms, let’s increase threshold. You can set threshold to 
whatever you want, but for the sake of this project, shoot to make 
it a manageable 25 percent. To find the correct number, simply 
do the following calculations with your doubled dimensions: 

480 x 640 = 307,200 
0.25 x 307,200 = 76,800 

Now that you know how many pixels equal a quarter of the 
image, you can plug that number in the configuration file for 
motion: 

sudo nano /etc/motion/motion.conf 

Find threshold and edit the line to say 

threshold 76800 
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Now motion will trigger an event when you wave your hand 
in front of the camera—but not when a bug flies around on the 
other side of the room. 

The rest of the configuration settings for motion are certainly 
worth checking out and experimenting with, though doing so 
would be a bit outside the scope of this book. If you’re inter¬ 
ested in learning about the various settings, you can find a 
good reference here: www.lavrsen.dk/foswiki/bin/view/Motion/ 
ConfigFileOptions. 

STREAMING MOTION REMOTELY 

Viewing motion remotely operates much like viewing it locally 
on the Pi. Once it’s set up, you can stream what motion sees 
from any web-enabled device, be it a phone, a laptop, or even 
another Pi. Before you can do this, however, there are a few 
things you’ll need. 

First of all, you must enable remote streaming with motion. 
Start by opening motion.conf: 
sudo nano /etc/motion/motion.conf 

Once the configuration file is open, find the stream_localhost 
line. This value determines whether or not motion allows remote 
streaming. To enable streaming, change it to read 

streamjlocalhost off 

With streaming now enabled, you just need to find the Pi’s IP 
address so that you can connect to it. This is easy to do by typing 
the following at the command line: 

hostname -I 
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The output the command gives is the IP address of the Pi. 
Armed with this knowledge, you can now stream motion from a 
remote device. Making sure that the Pi is connected to the Inter¬ 
net, open the web browser on your device and type the Pi’s IP 
address in the address bar, followed by the port number sosi. 
The bar should resemble the following: 

http://192.165.76.3:8081 

Start motion: 


Enter the IP address and your browser should look just as it 
did when you viewed motion locally on the Pi. 


GOING HEADLESS 

Streaming motion headlessly—that is, without connecting the Pi 
to a monitor or keyboard—is simple enough to do. You know how 
to watch the stream from motion remotely at this point, which is 
half the battle. The other half is getting motion to run while the 
Pi is headless. 

There are two ways to accomplish this. The first is to connect 
to the Pi from a remote computer via SSH and tell motion to run. 

The other way is to have motion run automatically when the 
Pi powers up. One of the best ways to do this is by using crontab. 
The crontab (short for cron table) file is used for scheduling dif¬ 
ferent commands and programs so that they run automatically 
without the system administrator having to enter the same com¬ 
mand every day. 

To use crontab, first open the file for editing: 
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pi@raspberrypi : / S sudo crontab -e 
no crontab for root - using an empty one 

Select an editor. To change later, run 'select-editor'. 

1. /bin/ed 

2. /bin/nano <-easiest 

3. /usr/bin/vim.tiny 

Choose 1-3 [2]: | 

FIGURE 3.5: Selecting an editor for crontab 

When you first open crontab, the system will ask you which 
text editor you wish to use. At the prompt, select nano by typing 
2 and pressing Enter. The crontab file should then open in the 
command window. 

Since you want to run motion at startup and not at any specific 
time, we will ignore the traditional scheduling format. Instead, 
scroll to the bottom of the file and enter the following: 

greboot motion 

Now motion will run every time you boot up your Pi. Save your 
edit and exit the file. 

To test and make sure the changes you made work, reboot 
your Pi: 


Then try to stream motion via the web browser. If everything is 
operational, you should see the feed from motion in the window 
without having to start the program manually. 
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Machine Learning: 
Identifying People 
in Images 


So you’ve posted a photo to your favorite social media site 
and it suddenly puts little boxes around each person’s face and 
asks you to tag who they are. The first part—recognizing there’s 
a face in the image—is facial detection. The second part is facial 
recognition—putting a name to a face. The reason the site is ask¬ 
ing you to name the people in the image? That’s so it can learn 
to recognize that face in other pictures. Next thing you know, 
it’ll be auto-suggesting people in other pictures—Facebook, for 
example, is learning faces. 

Now that you have a smart camera set up, we’re going to 
add both facial detection (“there is a human face in the picture”) 
and facial recognition (“and that face is my face!”). There are two 
steps to this. First, we’ll install software that can detect whether 



there is a face-shaped object in the image (facial detection); then 
we’ll try to extract that object and see whose face it is (facial 
recognition). 

The same software does both—the first operation is quick, 
and the second takes a little more effort. Just as with the pre¬ 
vious projects, the software for doing the hard work is easy to 


OpenCV 

OpenCV, also known as the Open Source Computer Vision 
Library (at https://opencv.org), is a library of over 2,500 
image-handling algorithms. They can be used for everything 
from interactive art, to mines inspection, to stitching maps 
on the web or to advanced robotics. These algorithms can be 
used to detect and recognize faces, identify objects, classify 
human actions in videos, track camera movements, track mov¬ 
ing objects, extract 3D models of objects, produce 3D point 
clouds from stereo cameras, stitch images together to produce 
a high-resolution image of an entire scene, find similar images 
from an image database, remove red-eye from images taken 
using flash, follow eye movements, and recognize scenery and 
establish markers to overlay it with augmented reality. 

In short, it’s free industry-grade software that is easily set up 
via Python. We’ll use OpenCV to do facial detection as well as 
facial recognition. If you decide to delve deeper into image pro¬ 
cessing, the best part is that OpenCV can do a lot more than 
just faces. If you want to dig deeper on the background theory 
and advanced capabilities of machine learning as it applies to 
images, you can read up on its full abilities at https://opencv.org. 
We’ll work off their specific section on facial recognition here: 

https://docs.opencv.Org/2.4/modules/contrib/doc/facerec/ 
facerec tutorial.html 

V _ J 
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install. It’s a package called OpenCV, and you probably already 
installed it in Chapter 1. But if not, install it with 

sudo apt-get install opencv 
pip install numpy 

(Numpy is a numerical library that is required by and speeds up 
OpenCV.) 

We’ll pull out the core parts of the tutorial for our specific 
SpyPi work. To begin, we’ll get going with the core functionality 
for your Pi system. After that, if you want to explore deeper, you 
have all the tools you’ll need in place to do so. 


STEP 1: QUICK FACE BOXING 

Let’s start by downloading a ready-to-go facial recognition script 
(from a user named Shantnu Tiwari, username shantnu) that finds 
faces and draws boxes around them in real time. Navigate to this 
website from your Pi: 

https://gitliub.com/sliantnu/Webcam-Face-Detect 

Click the Download button to download the zip file. Then, in 
your terminal, unpack it: 

Then, to run it: 

cd Webcam-Face-Detect-master 
python webcam.py 

Figure 4.1 shows the code in that webcam, py file. 
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That’s it. You will now see the view from your webcam, and 
as the software detects a face, it will draw a box around the face. 
The video is choppy, with slight delays between what you do and 
what it shows; this is because it takes a while for the software to 
run the facial detection. We’ll tackle what the software is doing 
and add coolness to it, but this script is a great way to show how 
easily you can do facial recognition on the Pi. 


NOTE Shantnu Tiwari, who provided this initial face-detection 
script, has other code bits you can run on his GitHub site (https:// 
github.com/shantnu/ ). He runs the website pythonforengineers 
.com, which includes the free online book Python for Scientists 
and Engineers, as well as other ebooks he’s written. 
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Figure 4.2 shows the first result, the author “found” by the 
software. 



FIGURE 4.2: Author’s face found and boxed by SpyPi 

It runs without you needing to understand it, but where is 
the fun in that? Let’s go deeper. First, look at the script (click 
it to open it in an editor, or if you use the terminal, type less 
webcam.py). The script is very short and written in Python; it loads 
the opencv library that handles the image processing as well as 
an XML file that contains mathematical definitions created by 
researchers that define what a computer will recognize as “a 
face.” The script then calls up the web camera and runs a loop to 
apply the face-finding algorithm. 

The first five lines of text set up the OpenCV package and 
the XML file. The while True: line says “loop forever.” The lines 
marked with # are comments that describe what the following 
lines in the code do. First the script captures a frame; then it 
draws a rectangle around the frame and displays that frame in 
the rectangle. The final two lines do cleanup. If you press Q at 
any point during the demo, the program exits, which is handy 
(and means you don’t need to press Ctrl-C as in earlier chapters). 

This script is good enough to tell a person from a stuffed 
animal or pet, as you can see in Figure 4.3. The accuracy of facial 
recognition software (even in industrial uses) is still a work in 
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progress, so we’ll look at the sorts of errors you can expect, 
well as let you try different methods. 



FIGURE 4.3: Face-finding test: person vs. stuffed animal. The software cor¬ 
rectly found the person and did not match on the stuffed animal. 


An interesting point about any image detection is that you 
get to choose between a false positive—it matches a face even 
if one isn’t there—and a false negative—there’s a face but it 
doesn’t find it. Let’s look at some image examples of this basic 
face-finding problem. Figure 4.4 contains both a false positive—it 
“finds” a face in some wall tiles that isn’t really a person—and a 
false negative—it fails to find the author’s face. 



It’s also worth noting that software relies on patterns, so if the 
only thing in the field of view has some face-like aspects, such 
as eyes, it might find a match. If a stuffed animal is the only thing 
in the camera’s field of view, it’s not surprising if the software 
sometimes (but not always) tries to tag it as a face. We do this as 
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humans—draw any two dots plus a line, and we’ll see a face. This 
phenomenon, called pareidolia, means humans tend to see faces 
in objects. Computer code can have the same problem. A classic 
example is the face seen in an outlet (Figure 4.5). 



FIGURE 4.5: Pareidolia—seeing faces in inanimate objects 

We can run into this phenomenon with software, particularly 
if we’re trying to fool it. The software doesn’t know what a “per¬ 
son” or “object” is, and it’s looking for patterns. If you mess up the 
patterns—put on Juggalo makeup, for example—you can confuse 
facial recognition. Likewise, an object such as a toy made to look 
like it has a face might find the software latching onto part of its 
face-like structure to falsely match a face. 
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Either way, you now have real-time facial detection working 
on your SpyPi. Let’s build on this to do more. 


Face Algorithms 


OpenCV can automatically find a face-like object using one of 
two built-in methods. Why two? One method (the Haar feature) 
doesn’t find faces easily, but when it finds a face you can trust 
that it’s actually a face (which means fewer false positives). The 
other method (LBP) is faster and finds more faces, but it also 
is more likely to tag something as a face that isn’t really a face 
(which means more false positives). Since the code we started 
with uses Haar, we’ll keep going with that. 


A LITTLE IMAGE THEORY 

How do you recognize someone? The most common human way 
is to notice a specific feature or set of features that identifies a 
specific person: that person with the dark hair and big nose, that 
other person with the eyeglasses and red lips. We latch onto 
these for quick identification. 
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When you do this with a group of people you’ve just met, 
you compare them to one another. It’s no good saying “she’s the 
one with dark hair” if everyone in the group has dark hair, so 
you make quick comparatives for just that group—the one red¬ 
head, the one with the narrow eyes, the one with glasses. This 
kind of quick sorting out of a group is the Eigenfaces method 
in OpenCV—within a group of known photos, it's how you tell 
one from another. The downside of this approach is that you can 
later get confused if a person changes her hairstyle or takes off 
his glasses (Clark Kent style!), or even if we see that person in 
different lighting. 

More often, you'll want to make a list of features that let you 
distinguish any face from the others: not just “the person with 
glasses” but a checklist—“Glasses: yes/no. Dark hair: yes/no” and 
so on. Then each person has a profile of features that you can 
use to compare with others. That’s the Fisherfaces method in 
OpenCV. Sure, the computer isn’t smart enough to know what 
“glasses” or “hair” are, but it can teach itself that the “dark blob 
at the top of the face-shape” is different from the “light blob at 
the top of the face-shape’’—and that’s practically the same thing 
for our purposes. 

Finally, you can go full-on machine thinking and instead of 
extracting components of faces to make a checklist, you let the 
computer do a pixel-by-pixel mapping of how faces change as 
you scan left to right and up and down. So where we see an “eye,” 
the computer just says, “There is a white zone with a non-white 
dot in the middle.” That’s the Local Binary Pattern Histogram 
(LBPH) method in OpenCV. 

In Python, these modules are called as follows: 

Eigenfaces: createEigenFaceRecognizer() 

Fisherfaces: createFisherFaceRecognizerO 

Local Binary Patterns Histograms: createLBPHFaceRecognizer() 


Machine Learning: Identifying People in Images 



These are the three different underlying algorithms available 
in OpenCV, each with its own pluses and minuses. We’ll be using 
LBPH, but (as you’ll see) you can change just one line in the code 
to use the other algorithms. 

STEP 2: TRAINING YOUR CAMERA 

Once you have an algorithm, you need to train your camera with 
the set of faces you think it will see. For example, if you want the 
camera to identify any visiting students and you have a set of 
student photos, you first make the software aware of what the 
students look like; then the camera can not only detect that a 
face exists but tag it and label it as whose face it is. 

Now that your camera can detect a face, we want to make it 
tell you whether that’s a specific known face, thus creating an ID 
system. To do this, you'll need an image of any and all faces you 
want it to recognize so that you can “train” your camera system 
to spot specific people. 

So, take a bunch of pictures of the faces of people you want 
to enter in its system. You'll then train the system by loading 
those in. Once trained, the program can then automatically rec¬ 
ognize known individuals. 

An untrained system can still put a rectangle around a face 
and say “There is a face,” but for identification purposes, training 
it is useful. 

To get this started, first you’re going to modify the previous 
code to make it easier to rework. We already did this and put it 
up on GitHub at 

https://gitliub.com/sandyfreelance/SpyPi 

Go get it and put it on your Pi, and we'll walk through what it 
does. This code will 1) display a new image every second, 2) draw 
rectangles around faces, 3) match any faces it “knows,” and 4) 
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let you also train it by telling it to add a face to its memory. The 
opening lines just set up the variables and load OpenCV and 
other programs: 



face.recog = cv2.createLBPHFaceRecognizer() 





After that, the code asks if you want to reload previously 
saved “known” faces: 



Enough preamble. 


STEP 3: IDENTIFYING WHOSE FACE 
YOU FOUND 

The meat of the code starts with the rectangle-drawing functions 
from before and adds a little face recognition. First, we use the 
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same routines from the previous code that grab an image and 
draw a box around it: 

def detect_face(vid_cap, all_faces, face_recog, face_count): 

# Capture frame-by-frame 
ret, frame = vid_cap.read() 

gray = cv2.cvtColor(frame, cv2.C0L0R_BGR2GRAY) 
faces = faceCascade.detectMultiScale( 
gray, 

scaleFactor=l.l, 

minNeighbors=5, 

minSize=(30, 30), 

flags=cv2.cv.CV_HAAR_SCALE_IMAGE 

) 

# Draw a rectangle around the faces 
for (x, y, w, h) -in faces: 

cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2) 
# also can put a trigger here 

What does the comment #also can put trigger here mean? This 
is where you can put code that triggers when any face is found, 
in case you want to make an automatic door opener or something 
similar. It’s just a comment indicating that this is a good place to 
put instructions for facial detection (before they are recognized). 

Now for the next bit. If we can draw a rectangle around a 
face, we can extract just the face and then compare it with any 
previous faces we might have stored in a list. We always convert 
faces to grayscale, which removes color imbalance concerns (and 
is also the format OpenCV expects). So do this: 

# extract just the face as its own image 
thisface = frame[y:y+w, x:x+h] 

grayface = cv2.cvtColor(thisface, cv2.C0L0R_BGR2GRAY) 

Here’s the fun part. It’s only one line that compares a new 
face found with “known” faces, using OpenCV’s .predict method: 

id = face_recog.predict(grayface) 
print "Found face number",id[0] 
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And to close off this part, we again display it onscreen and 
return any faces we found to our main calling program: 

# Display the resulting frame 
cv2.imshow('Video', frame) 
return faces_found 

That wraps up the code for the new subroutine. Next we’ll put 
in some bookkeeping and add in the OpenCV functions that do 
the recognition. We again use a “keep looping until we tell you 
otherwise” construct, and we allow for two different keypresses: 

Q quits the program. 

S saves the currently rectangle-highlighted face as a “known” 
face. 

So we train the system by pressing S when a known person is 
there, and then the system automatically matches it to any new 
faces found. You could move the training to its own program, 
but having the script allow you to both train and predict made it 
smaller and easier. 

while True: 

faces_found = detect_face(vid_cap, all_faces, face_recog, 
face_count) 

checkme = cv2.waitKey(100) 
if checkme & 0xFF == ord('q'): 
print "Exiting" 

if checkme & 0xFF == ord('s') and len(faces_found) > 0: 

# assumes we are storing one and only one face 
print "Storing a new face" 


all_faces.append(faces_found[0]) 
labels = range(face_count) 

face_recog.train(all_faces,numpy.array(labels)) 

Yet again, all the real work is being done in the OpenCV 
.trainQ method call. You pass it a list of faces you want it to 
learn, and it handles the rest. 
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Now to wrap everything up by cleanly closing out our win¬ 
dows (once the user presses the Q key), and then also saving 
any faces you marked as known (by pressing S, perhaps multiple 
times): 

# When everything is done, release the capture 

cv2. destroyAHWi ndows () 

# also save any 'trained' faces 
outfile = open(savefile,'wb') 
pickle.dump(all_faces, outfile) 
outfile. closeQ 

How well does it work? In Figure 4.7 we pressed the S key to 
tell the software to store this fine face it found. 



FIGURE 4.7: Training your SpyPi to recognize me (runtime on left, video 
capture on right) 

Then I'll move around and see if it can still find me. It can! It 
keeps listing “Found face number 0” in its runtime window. You 
can also modify the code and replace that line with, well, what¬ 
ever you want your Pi to do. You can have it trigger an alarm, send 
a message, and many other things, by combining other Jumpstart 
guides with this one. We provide the face recognition; you add 
the hardware. 

So you can see how it saves trained data, Figure 4.8 shows 
a new run that loads the automatically saved data from the pre¬ 
vious run. It still recognizes me, because now I’m in its saved 
trained data set. 
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And, to complete the tests, we check whether I can fool it 
with a cunning disguise. 



FIGURE 4.9: Even using a Santa hat does not fool the SpyPi face 
recognizer. 


TRIGGERING AND FUTURE STEPS 

Once you have categorized found faces as either known or 
unknown, you can add triggers to your code to make your Pi 
react. Maybe you want it to flash an LED (or lock a door) if it sees 
a face it doesn’t recognize. Or flash an LED (or open a door) if it 
sees a face it does know. With the previous code and a little work 
in Python, you can add in these “triggered” effects by editing the 
code provided. Just take the line that prints “face not found” and 
have it call a Python routine to carry out your automated wishes. 
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Using the Pi Ribbon Camera 

As usual, if you are using the Pi ribbon camera instead of a USB 
web camera, you just need to change a few lines in our examples 
to repoint our code to the correct camera. We’re not going to 
rewrite all the code here, but you should be able to figure out 
the mods easily enough. To capture with a web c 
using code with this type of call: 


from pit 


v2.imwriteCtest.jpg 1 ,image) 

e the Pi ribbon camera instead, you u 

= PiCameraO 

capture('image.jpg') 


5 constructs of this 


The picamera package is installed on the Pi automatically, and 
full documentation is at http://picamera.readthedocs.io/. 


Put simply, once code can find a face, it can act on that knowl¬ 
edge. Building this system into a security system is one possi¬ 
bility. Saving only images of faces is another surveillance-type 
activity you can use it for. There’s not a lot of use for, for example, 
one week of raw video, even if you can find faces. However, if you 
capture faces only when they appear and put that together, you 
have a video log of all the people who traveled past your system. 

Add in our earlier motion-detection and time-lapse work, and 
you have a system that can capture time-lapse images only when 
there’s activity involving people with faces: 

* Have it take pictures only when there’s motion (Chapter 3). 
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* For those pictures, if there’s a face, have it save the image 
of the face (this chapter). 

* Build those into a time-lapse movie of “people who visited 
us” (Chapter 2). 

Faces are only the start. Using the same OpenCV libraries, 
you can perform recognition of other objects, edge detection, 
real-time color filtering or background subtraction, conversion to 
black and white, motion trails, and all sorts of other image manip¬ 
ulation (see https://pythonprogramming.net/loading-images- 
python-opencv-tutorial/)—even converting images in near real 
time to artistic styles like Van Gogh or A-ha/Take-On-Me stylings 
(see https://larseidnes.com/2015/12/18/painting-videos-with- 
neural-networks/). 

At this point, armed with a Pi and some simple Python soft¬ 
ware, you can bend image reality to do everything from recogni¬ 
tion to alteration. Enjoy capturing and manipulating visual data! 
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